1. Field of the Invention
A large quantity of labor is required in the work of manually classifying and putting in order an infinite number of images present in the database, in the computer, or on the network and of attaching key words for retrieval with hands.
2. Description of the Related Art
Therefore, there are proposed an automatic image classification apparatus and an apparatus of automatically adding keywords for image retrieval.
As an example of the automatic image classification apparatus, there is available Jpn. Pat. Appln. KOKAI Publication No. 11-328422. Furthermore, an apparatus of automatically adding key words for retrieval, there are available Jpn. Pat. Appln. KOKAI Publication No. 8-249349, Jpn. Pat. Appln. KOKAI Publication No. 10-49542, and Jpn. Pat. Appln. KOKAI Publication No. 10-55366 and the like.
In the beginning, in Jpn. Pat. Appln. KOKAI Publication No. 11-328442, there is proposed an apparatus of automatically classifying, for example, natural images and artificial images by using features extracted from images by sampling.
Although there is a possibility that this apparatus which is capable of classifying images functions well depending upon the features distributed uniformly over the whole image, but the apparatus is not sufficient for classifying objects locally present in images.
That is, it is necessary to devise means for clipping only object regions in order to classify objects.
Such devise is introduced in Jpn. Pat. Appln. KOKAI Publication No. 8-249349, Jpn. Pat. Appln. KOKAI Publication No. 10-49542 and Jpn. Pat. Appln. KOKAI Publication No. 10-55366.
In Jpn. Pat. Appln. KOKAI Publication No. 10-49542, there is proposed that a region is divided by using a change in color or luminance in images in the beginning, and the feature of each of the divided regions is extracted, followed by analyzing a topological relationship between regions and referring to a dictionary so that the key words for retrieval is automatically added to images.
Furthermore, in Jpn. Pat. Appln. KOKAI Publication No. 8-249349, Jpn. Pat. Appln. KOKAI Publication No. 10-55366, there is proposed an apparatus of dividing an image into regions by using a change in color or luminance in images, subjecting divided regions to integration processing to transform the regions into a concept of an upper layer so that a high degree of key word is automatically added to the image as a key word.
These apparatuses are intended to automatically add key words on the basis of features of an object in the images, and the apparatuses use a knowledge dictionary such that a blue sky is disposed at an upper section of the image and has a blue color, or the like.
Consequently, there remains a problem such that how a parameter value for classification should be set in the case where classification which is not seen in knowledge dictionary is desired.
As means for coping with any circumstances, there is available a classification apparatus provided with a supervised learning. As such example, there is available Jpn. Pat. Appln. KOKAI Publication No. 11-34450.
Hereinafter, a conventional example shown in Jpn. Pat. Appln. KOKAI Publication No. 11-34450 will be explained in detail for some time.
FIG. 6 is a block diagram showing one example of a conventional classification apparatus provided with a supervised learning.
In FIG. 6, reference numeral 1001 denotes an image input and display section for inputting and displaying, for example, a wafer image.
Furthermore, reference numeral 1002 denotes a defect image selection section with which an operator monitors a displayed image and selects a defect image.
Furthermore, reference numeral 1003 denotes a calculation section for comparison with a reference image for calculating a difference from the reference image.
Furthermore, reference numeral 1004 denotes a defect region extraction section for conducting a threshold value processing associated with the calculation result of the calculation section for comparison with the reference image to extract a defect image.
Furthermore, reference numeral 1005 denotes a teacher signal (category signal) adding section for adding a category name which is a teacher signal at the learning step.
Then, reference numeral 1006 denotes a feature extraction section for extracting features of the extracted defect region.
Furthermore, reference numeral 1007 denotes a classification parameter learning section for conducting learning of the classification parameter so that the learning data of the attached with a teacher signal is well classified.
Furthermore, reference numeral 1008 denotes a category determination section for determining which of the category the defect region belongs to.
Furthermore, reference numeral 1009 denotes a category name adding section for adding a determined category name to the defect image.
Next, an operation thereof will be explained.
FIGS. 7A and 7B are flowcharts shown for explaining an operation of a conventional classification apparatus provided with a supervised learning.
At first, the flow of the learning step will be explained.
In the beginning, at the image input and display section 1001, for example, a wafer image is input and displayed (step ST 1001).
Next, at the defect image selection section, an operator selects a defect image for the training data (step ST1002).
Subsequently, at the calculation section 1003 for comparison with the reference image, a difference between the selected defect image and the reference image is calculated (step ST1003).
This calculation result is subjected to the threshold value processing at the defect region extraction section 1004 to extract the defect region (step ST1004).
The operator adds the category name which is a teacher signal to the defect region at the teacher signal (category name) adding section 1005.
Next, at the feature extraction section 1006, the features is extracted from the extracted defect region at the feature defect section 1006.
Subsequently, at the classification parameter learning section 1007, the classification parameter is learned so that the learning data attached with the teacher signal so that the learning data attached with the teacher signal is well classified (step ST1007).
Thus, the learning step is completed.
Next, the classification step will be explained.
A rough flow of the learning step is approximately the same as the learning step, so that only different processing section will be explained.
In the beginning, at the learning step, the defect image for training data is selected. However, at the classification step, all the classification object images become an object (step ST1009).
Furthermore, naturally, it is not required to add the teacher signal.
Instead, a category determination is made to determine which of the category the signal belongs to (step ST1013). The classification step will be completed by adding the name of the category (step ST1014).
As shown above, an example of the classification apparatus provided with the supervised learning has been explained according to Jpn. Pat. Appln. KOKAI Publication No. 11-34450.
By the way, Jpn. Pat. Appln. KOKAI Publication No. 11-34450 discloses a proposal on a defect classification apparatus in a defect inspection of a semiconductor wafer or the like.
Consequently, such defect classification apparatus is capable of detecting the position of defects from a difference between images with defects and reference images. In the case where an object region such as general images without any reference images is classified, it is extremely difficult to detect only regions to be classified.
However, since the device for automatically adding key words for image retrieval, and an automatic image classification apparatus are constituted in the above manner, so that the detection of the classification target region becomes a large issue.
As one method of detecting the classification target region, there is available a method of selecting and discarding and integrating each region which is divided by referring to the knowledge dictionary. In the case where it is desired to make a classification which cannot be found in the knowledge dictionary, there remain a problem as to how the value of the classification parameter value can be set.
A learning type classification apparatus can solve this problem, and can store what is learned as knowledge.
A structure of the classification apparatus provided with a supervised learning according Jpn. Pat. Appln. KOKAI Publication No. 11-34450 is convenient for inspecting a deviation from the reference image. However, it is extremely difficult to clip classification target region at the learning phase from general images.
As a consequence, work is considered to manually clip classification target region steadily at learning phase. However, such work is not preferable because a large quantity of labor is required.